REPORT DOCUMENTATIOni PAGE 


Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instruction; 
the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including 
Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, 


AFRL-SR-BL-TR-01- 


and reviewing 
ir Information 


1. AGENCY USE ONLY 


I 2. REPORT DA.TE 


I ire HI¥U uHlt:5 UUVERED 

15 FEB 97 - 29 NOV 99 


4. TITLE AND SUBTITLE 

The Design, Implementation, Application, and Dissemination of an Evolving 
Expression-Linkage Genetic Algorithms 

5. FUNDING NUMBERS 

F49620-97-1-0050 

6. AUTHOR(S) 

David E. Goldberg 


7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 

8. PERFORMING ORGANIZATION 

Department of General Engineering 

The University of Illinois at Urbana-Champaign 

117 Transportation Builiding 

104 S. Mathews Avenue 

Urbana. IL 61801 

REPORT NUMBER 

9. SPONSORINGIMONITORING AGENCY NAME(S) AND ADDRESS(ES) 

10. SPONSORING/MONITORING 

AFOSR/NM 

AGENCY REPORT NUMBER 

801 N. Randolph Street Room 732 

Arlington, VA 22203-1977 

F49620-97-1-0050 



12a. DISTRIBUTION AVAILABILITY STATEMENT 

APPROVED FOR PUBLIC RELEASE, DISTRIBUTION UNLIMITED 



13. ABSTRACT (Maximum 200 words) 

AFOSR Grant No. F49620-97-1-0050 (Evolving Expression-Linkage GA) has come to a close and this report details project 
accomplishment. Particularly notable was the invention of a new competent GA, the Bayesian Optimization Algorithm (BOA' 
that works by budding and using a probabilistic model of the best solution points seen so far. BOA scales well and solves ver 
hard problems quickly reliably and accurately. Another notable achievement was the completion of a Erick Cantu-Paz's 
magisterial PhD dissertation entitled Designing Efficient and Accurate Parallel Genetic Algorithms." This thesis provides the 
first scaling laws that show why GAs are so easy to make parallel and how to do it better. These findings and 
accomplishments are discussed in the final project report together with others, including the development of a new theory of 
hybrid optimization and a. new theory of effective lime utilization. The report also details the affiliation and support of 26 
researchers with the project, the publication, acceptance, or submission of 65 publications acknowledging AFOSR support, 
and numerous interactions and transitions. 


20011212 100 



17. SECURITY CLASSIFICATION 
OF REPORT 


18. SECURITY CLASSIFICATION 
OF THIS PAGE 


19. SECURITY CLASSIFICATION 
OF ABSTRACT 


15. NUMBER OF PAGES 
17 


16. PRICE CODE 


|20. LIMITATION OF ABSTRACii 


Standard Form 298 (Rev. 2-89) (EG) 

Prescribed by ANSI Std. 239.18 

Designed using Perform Pro, WHSIDIOR, Oct 94 











Final Report 

The Design, Implementation 
Application, and Dissemination of an 
Evolving Expression-Linkage Genetic Algorithm 
AFOSR Grant No. F49620-97-1-0050 

15 February 1997 to 30 November 1999 


David E. Goldberg 

Department of General Engineering 
liliGAL Report No. X0007 (Limited Circulation) 
March 2000 


Department of General Engineering 
University of Illinois at Urbana-Champaign 
117 Transportation Building 
104 South Mathews Avenue 
Urbana, IL 61801 




Final Report for 

The Design, Implementation, 

Application, and Dissemination 
of an Evolving Expression-Linkage Genetic Algorithms 
AFOSR Grant No. F49620-97-1-0050 

15 February 1997 to 30 November 1999 

David E. Goldberg 
Department of General Engineering 
The University of Illinois at Urbana-Champaign 
117 Transportation Building 
104 S. Mathews Avenue 
Urbana,IL 61801 


Abstract 

AFOSR Grant No. F49620-97-1-0050 (Evolving Expression-Linkage GA) has come to a close 
and this report details project accomplishment. Particularly notable was the invention of a new 
competent GA, the Bayesian Optimization Algorithm (BOA) that works by building and using a 
probabilistic model of the best solution points seen so far. BOA scales well and solves very hard 
problems quickly, reliably and accurately. Another notable achievement was the completion of a 
Erick Cantii-Paz’s magisterial PhD dissertation entitled “Designing Efficient and Accurate Parallel 
Genetic Algorithms.” This thesis provides the first scaling laws that show why GAs are so easy 
to make parallel and how to do it better. These findings and accomplishments are discussed in 
the final project report together with others, including the development of a new theory of hybrid 
optimization and a new theory of effective time utilization. The report also details the the affiliation 
and support of 26 researchers with the project, the publication, acceptance, or submission of 65 
publications acknowledging AFOSR support, and numerous interactions and transitions. 


1 Objectives 

The augmented objectives (see previous project reports) of Grant No. F49620-97-1-0050 (Evolving 
Expression-Linkage GA) were as follows: 

• Design and implement an evolving expression-linkage genetic algorithm (eelGA). 

• Apply the eelGA to a problem of Air Force interest. 

• Transfer this technology to Air Force personnel and vendors. 

• Improve the efficiency of GAs by four means: 

1. parallelization techniques. 
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2. time utilization methods. 

3. evaluation relaxation. 

4. effective hybridization. 

Significant progress was made in accomplishing each of these objectives, and this progress is briefly 
reviewed in the next section. 

2 Status of Effort 

The project has moved ahead with progress in each of the objective areas: 

• Development of effective competent GAs. 

• Development of effective efficiency techniques. 

• Dissemination of the results to Air Force personnel and tackling problems of importance to the 
Air Force. 

Each of these areas is briefly reviewed in what follows. 

2.1 Competent GAs 

The major task of the project was to advance the cause of competent genetic algorithms—GAs that 
solve hard problems, quickly, reliably, and accurately. Progress in this area over the course of the 
project has been substantial and included work on 

1. The Bayesian optimization algorithm (BOA) 

2. The linkage identification through non-linearity checking and monotonicity detection algorithms 
(LINC/LIMD) 

3. The development of OmeGA, the ordering messy genetic algorithm 

Martin Pelikan’s (AFOSR-sponsored) work on the Bayesian Optimization Algorithm (BOA) is perhaps 
the most notable achievement of the project (and the subject of a follow-on project). Simply stated, the 
code builds a Bayes net model of the best points, using this model in place of crossover, mutation, and 
other genetic operators to speculate on the best points in the search space. Because of the generality of 
the Bayes net assumption, BOA can quickly search for optimal solutions under more general conditions 
than can a linkage-oriented solver. This is quite exciting, and the continuation of this work was proposed 
and funded by AFOSR and is currently ongoing. 

Professor Masaharu Munetomo, a visitor from Hokkaido University in Japan, and the PI developed 
two new techniques for linkage identification and exploitation, the so-called LIMD and LING GAs 
(Linkage identification through monotonicity detection and nonlinearity checking). These results may 
be viewed as alternatives to Kargupta’s gene expression messy GA that also derived from IlliGAL 
activities sponsored by AFOSR. 

Dimitri Knjazew, a visiting scholar from Dortmund, Germany, and the PI developed OmeGA, the 
ordering messy genetic algorithm for solving hard problems over permutations. Using previously sup¬ 
ported fast messy GA technology, the researchers have come up with a way to solve difficult scheduling 
and sequencing problems quickly, reliably, and accurately. 
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2.2 GA efficiency enhancement 

Progress in the area of GA efficiency is important, because even competent GAs should be expected 
to require significant computational resources. This project has pursued four types of speedups: 

1. parallelization 

2. time utilization 

3. evaluation relaxation, 

4. hybridization 

Our first foray into the realm of efficiency enhancement came with Miller’s groundbreaking thesis 
on optimal objective function sampling in the presence of noise. Miller’s work used practical theory 
and careful experiment to demonstrate that GAs tend to prefer quick and noisy/dirty evaluation to 
slow and accurate in the solution of problems in a given time. Scaling laws were derived for the optimal 
sample sizing, and these were confirmed in practice. Follow-on work is now extending this results to 
function surrogates, optimal finite-elements and differential equation grid sizing, and other types of 
error-computation-time tradeoffs. 

Among the most important accomplishments in this area was the recent completion of Erick 
Cantu-Paz’s magisterial PhD dissertation Designing Efficient and Accurate Parallel Genetic Algorithms 
(UIUC, Computer Science, 1999). Parallel GAs have long been used in practice, but this usage has 
been largely without benefit of even the most rudimentary scaling laws. In one fell swoop, Cantii-Paz 
rectifies the situation with both bounding and exact models of parallel GA performance and design. 

An important first model of global-local hybrids (Goldberg & Voessner, 1999) was developed to try 
to identify the optimal division between global searchers (like GAs) on the one hand and local searchers 
(those of OR and traditional optimization) on the other. 

First theory on optimal time utilization (Goldberg, 1999) was derived as part of this project, and 
experimental work and theoretical refinement are continuing to try to ere performed this past year as 
a result of AFOSR funding 

2.3 Dissemination of results to the AF and tackling AF-related projects 

Dissemination of these results to the Air Force (and others) has been improved by placing greater 
emphasis on getting pilot codes up sooner. Pelikan’s BOA, Harik’s extended compact GA, and a 
version of Wilson’s XCS have been made available (IlliGAL Technical Report Nos. 99011, 99016, and 
99021). Additionally, at a program review meeting at WPAFB, I proposed a Workshop on Military 
Applications of Genetic and Evolutionary Computation at which it might be possible to give a tutorial 
on competent GAs and their application. Additionally, a meeting was held at WPAFB on July 8, 1999 
to investigate the application of the methods of this proposal and the next to molecular conformation 
problems with Dr. Ruth Pachter of WPAFB, 

Problems of military and Air Force interest have been pursued with other researchers throughout 
the course of this investigation, including antenna/absorber design, course-of-action selection in combat 
situations, design of molecules, scheduling applications, to name a few. Additional details are given 
under findings and transitions. 

Taken together these results represent success across the spectrum of goals originally anticipated 
in the project. Specific accomplishments and findings are summarized in the next section. 
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Speedup 



Figure 1: The speedup curve for rucister-slave parallelism is shown for three values of evaluation- 
communication time ratio 7 = 1,10,100 from Cantii-Paz (1999). 


3 Accomplishments and New Findings 

This section highlights the key accomplishments and findings of the project. 

BOA developed and published. Martin Pelikan has invented the Bayesian Optimization Algorithm 
(BOA), which builds and exploits a probabilistic model (a Bayesian network) of the population of best 
points seen by the algorithm. Because the Bayesian network is such a general representation, BOA has 
been found to solve a broad class of difficult problems, quickly, reliably, and accurately. First generation 
BOA code is available for Air Force use on the IlliGAL web site (http://www-illigal.ge.uiuc.edu), and 
work is continuing to improve BOA to handle hierarchical problems and alternative codings. 

Parallel GAs unlocked. Erick Cantii-Paz has completed an important dissertation on the analysis 
and optimization of parallel genetic algorithms. Because of this work, we can now predict and optimize 
solution quality under differing topologies, migration rate, and population size. The work has exceeded 
our initial expectations and should be viewed as a turning point in the science and art of parallel 
GAs. For example, his bounding calculation for the number of processors required in master-slave 
parallelization 

S'= 

where S'= the number of processors, f3 is the function evaluation time per individual, and T- is the 
communication time per individual, shows us that large numbers of processors can be used effectively 
whenever function evaluation time greatly exceeds communications time—a situation that occurs in 
most problems of practical interest. The optimal result can be seen as the maximum in the speedup 
curve of figure 1. The speedup is approximately linear until a critical value of processor count of 



This is good news and suggests that many practical applications can expect near-linear speedups as 
additional processors are added to the task, even if the form of parallelization is no more sophisticated 
than a simple master-slave configuration. 
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DOI Book completed. Tlie PI has completed a book entitled The Design of Innovation: Lessons 
from Genetic Algorithms and is now distributing chapters for review by others. The book brings 
together (1) the methodology, (2) the design theory, and (3) the competent GA designs that have been 
developed largely with AFOSR support. The book starts with a discussion of the Wright brothers’ 
analogy the PI has used over the used to discuss GA design and continues with sections on little 
models and patchquilt integration using dimensional analysis. Chapters on building blocks, the schema 
theorem, selection timing, mixing, problem difficulty, and decision making bring IlliGAL design theory 
to a technical audience. Chapters on competent GAs (like the fmGA, LLGA, BOA, LINC-GA and so 
on) and the connection of competent GA design to human innovation cap off the volume. The book 
should be published in the year 2000. 

LINC/LIMD GA, a new competent GA, designed. Professor Munetomo, a visitor from Japan, 
has invented two competent GAs: one called the LINC-GA (linkage identification by nonlinearity 
checking GA) and another called LIMD-GA (linkage identification by monotonicity detection). Both 
techniques are motivated by the idea of the detection of linkage groups through pairwise comparisons 
of alleles. Such a criterion is not adequate to detect the types of problems we are interested in (not 
all nonlinear problems are hard). Nonetheless, Dr. Munetomo has devised methods to rectify this 
difficulty. The method gives us one more tool in the arsenal of competent GA design, and it also 
is helpful in understanding similarities and differences between the various methods that have been 
established to date. 

Economics of hybridization model developed. The PI performed first analyses and experiments 
(with Dr. Siegfried Voessner, Engineering Economics and Operations Research, Stanford University) 
on a theory of hybridization economics. Most GAs in practice are hybrids between genetic algorithms 
and some local search technique, and such combinations have been quite successful at achieving good 
answers quickly, but there has been less inquiry into how to divide one’s effort between the different 
schemes, and in our work we have constructed a first theory of economic combination that suggests how 
to divide ones time between local and global search using minimal assumptions about the search space 
and the searchers. Figure 2 shows the theoretical vs. experimental results for a random global searcher 
used alone versus one used with a local search technique. In the particular case, theory predicts that 
the global random search should be superior to G4-L as is measured in practice. 

Time utilization analysis performed. Effective time utilization is demonstrated to be the missing 
link between crossover and mutation in a study sponsored by this project. The article “Using time 
efficiently: Genetic-evolutionary algorithms and the continuation problem.” (Goldberg, 1999), makes 
the point that salience distribution and signal-to-noise processing are the usual determinates of whether 
a building block is processed in parallel with others, or serially when SNR ratios are favorable. A 
bounding analysis suggests a dividing line between the two types of behavior that is quite dramatic. 

BOA, XCS, and ECGA code made available. Pelikan’s Bayesian Optimization Algorithm (BOA), 
Butz’s version of Wilson’s XCS code, and Harik’s Extended Compact GA have been published as C-I-+ 
code on the IlliGAL web site (http://www-illigal.ge.uiuc.edu). 

fmGA study of scheduling problems with random keys. The random keys coding is used to¬ 
gether with the AFOSR-supported fast messy GA (fmGA) to tackle difficult scheduling problems in 
the ordering messy genetic algorithm (OmeGA). Initial results are exciting and suggest that the com¬ 
bination of coding and fmGA work quickly, reliably, and accurately on target problems of considerable 
bounded difficulty. Recently completed results show that OmeGA scales to real scheduling problems. 
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Figure 2: The probability of not achieving solution as a function of allowable number of function 
evaluations for a global scheme only or global+local hybrid matches the model of Goldberg & Voessner, 
1999) 


XCS coded and applied to engine test problem. As part of cooperative work with the Na¬ 
tional Center for Supercomputing Applications, Stewart Wilson’s XCS genetics-based classifier system 
has being coded (by Martin Butz) and applied to an engine test problem for Caterpillar. It is fully 
operational and is now available for Air Force (and other personnel). 

ACS investigation underway. Martin Butz is also continuing his work on Stolzmann’s so-called 
anticipatory classifier system (ACS), The ACS learns new rules by minimizing the difference between 
what the system anticipates and what it observes. ACS has been used on small problems, and these 
efforts are to understand how to enable ACS-like techniques scale to real problems in classification. 

Optimal sample size work continued. Phillipe Giguere has continued Brad Miller’s important 
work on optimal sample sizing by looking at the case of sampled fitness functions in which sampling 
can result in the elimination of sampling error in a finite number of samples. In a sampled one-max 
problem, the small sample sizes suggested by the Miller bounds do not hold up, and the solution quality 
is surprisingly indifferent to variation of the number of samples. We were also surprised to observe the 
sensitivity of the analytical calculations to changes in the population sizing model. Additional work 
needs to be done, but this work shows that some care should be exercised in extrapolating the Miller 
work beyond its intended domain of application. 

Population sizing shown to be proper bound. The PI has collaborated with Gunter Rudolph at 
Dortmund University and have shown that the gambler’s ruin population sizing model we now use is 
a proper bound of the full Markov chains. This result should help persuade the theory community of 
the soundness of our approach. 
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Facetwise analysis of genetic programming initiated. Working with Una-May O’Reilly at the 
MIT AI lab, the PI has initiated the process of carrying over the method of his AFOSR studies to 
genetic programming. This work is critical, because (1) GP is the fastest growing segment of the field, 
and (2) GP related methods hold the greatest promise to solve deep, difficult problems in many fields. 
The first two papers resulting from this collaboration have been published. The first discusses different 
types of expression mechanisms and their relationship to where program material resides in a genetic 
program. The second discusses the relationship between building block salience and temporal sequence 
of processing. Understanding these two items sets the stage for a better understanding of the critical 
issues of population sizing and mixing, work that is continuing. 

Linkage learning code compressed and made available. Harik’s linkage learning GA has been 
published in an efficient compressed form. This permits us to use introns or non-coding segments of 
arbitrary length without arbitrarily large memory requirements. It also makes the code available for 
the first time to Air Force and other users. 

4 Personnel Supported 

This section details the individuals supported on these projects. 

4.1 Faculty supported 

Professor David E. Goldberg, the principal investigator, was supported on eelGA during the summer 
1999. 


4.2 Faculty affiliated with the project 

The following is a list of visiting faculty working on topics related to the project whose primary support 
comes from another source. Nonetheless, their supervision by Professor Goldberg was supported by 
AFOSR funds, and some travel or incidental expenses may have come from AFOSR monies. 

1. Professor Masaharu Munetomo (Hokkaido University) 

2. Professor Yuji Sakamoto (Muroran Institute of Technology) 

3. Professor Dirk Thierens (University of Utrecht) 

4.3 Graduate students supported 

The following is a list of graduate students supported. Unless otherwise indicated all students are or 
were affiliated with the University of Illinois. 

1. M. Pelikan (UIUC PhD CS) 

2. D. Knjazew (Dortmund University) 

3. L. Wang (UIUC, MS, ECE) 
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4.4 Graduate students affiliated with project 

The following is a list of students working on topics related to the project whose primary support comes 
from another source. Nonetheless, their supervision by Professor Goldberg was supported by AFOSR 
funds, and some travel or incidental expenses may have come from AFOSR monies. 

1. M. Butz (University of Wuerzburg, Germany) 

2. E. Cantii-Paz (UIUC, Supported by Fulbright) 

3. P. Giguere (UIUC) 

4. G. Harik (Michigan) 

5. J. Horn (UIUC) 

6. F. Lobo (Nova University, Lisbon) 

7. O. Mengshoel (UIUC) 

8. B. Miller (UIUC) 

9. A. Pereira (Nova University, Lisbon) 

4.5 Undergraduate students supported 

The following is a list of undergraduate research assistants who have been supported by AFOSR funds: 

1. J. Borgerson 

2. B. Dunn 

3. R. Gadient 

4. P. Gallego 

5. K. Garner 

6. A. Herzog 

7. M. Magin 

8. B. Sutton 

9. B. Wagus 

10. K. Zacarias 

5 Publications 

Publications submitted, accepted, and published over the course of the project are presented in this 
section. 
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5.1 Submitted 

Butz, M. V., Goldberg, D. E., k Stolzmann, W. (2000). Probabililty-enlianced predictions in the an¬ 
ticipatory classifier system. Proceedings of the 2000 International Workshop on Learning Classifier 
Systems. 

Butz, M. V., Goldberg, D. E., k Stolzmann, W. (2000). Investigating generalization in the anticipa¬ 
tory classifier system. Parallel Problem Solving from Nature. 

Knjazew, D., k Goldberg, D. E. (2000). Large-scale permutation optimization with the ordering 
messy genetic algorithm. Parallel Problem Solving from Nature. 

Pelikan, M., k Goldberg, D. E. (2000). Genetic algorithms, clustering, and the breaking of symmetry. 
Parallel Problem Solving from Nature. 

Pelikan, M., k Goldberg, D. E. (2000). Research on the Bayesian optimization algorithm. Parallel 
Problem Solving from Nature. 

Reed, P., Minsker, B., k Goldberg, D. E. (2000). Designing a competent simple genetic algorithm for 
search and optimization. Water Resources Research. 

Rothlauf, F., k Goldberg, D. E. (2000). Pruefernumbers and genetic algorithms: A lesson how the 
low locality of an encoding can harm the performance of GAs. Proceedings of EuroTel 2000. 

Stolzmann, W., Butz, M. V., Hoffman, J., k Goldberg, D. E. (2000). First cognitive capabilities in the 
anticipatory classifier system. Proceedings of the 2000 International Conference on the Simulation 
of Adaptive Behavior. 

5.2 Accepted but not published 

Butz, M. V., Goldberg, D. E., k Stolzmann, W. (in press). Introducing a genetic generalization pres¬ 
sure to the anticipatory classifier system, Part I: Theoretical approach. GECCO’2000 Proceedings 
of the Genetic and Evolutionary Computation Conference. 

Butz, M. V., Goldberg, D. E., &: Stolzmann, W. (in press). Introducing a genetic generalization pres¬ 
sure to the anticipatory classifier system. Part 2: Performance analysis. GECCO-2000 Proceedings 
of the Genetic and Evolutionary Computation Conference. 

Cantii-Paz, E. (in press). Implementing fast and reliable parallel genetic algorithms. In Chalmers, L. 
(Ed.) Handbook of practical genetic algorithms^ III^ Boca Raton, FL: CRC Press. 

Cantii-Paz, E., k Goldberg, D. E. (in press). Efficient parallel genetic algorithms: Theory and 
practice. Computational Methods in Applied Mechanics and Engineering. 

Goldberg, D. E. (in press). The design of innovation: Lessons from genetic algorithms, lessons for the 
real world. Technological Forecasting & Social Change. 

Goldberg, D. E. (in press). Genetic and evolutionary algorithms in the real world. Information 
Processing Society of Japan Magazine [In Japanese, N. Adachi, trans.]. (also IlliGAL 99013 in 
English) 

Goldberg, D. E., & Deb, K. (in press). Genetic and evolutionary computation in computational 
mechanics and engineering: Preface to a special edition. Computational Methods in Applied Me¬ 
chanics and Engineering. 
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Harik, G., k Goldberg, D. E. (in press). Linkage learning through probabilistic expression. Compu¬ 
tational Methods in Applied Mechanics and Engineering. 

Knjazew, D. & Goldberg, D. E. (in press). OMEGA—Ordering Messy GA: Solving permutation 
problems with the fast Imessy genetic algorithm and random keys. GECCO-2000 Proceedings of 
the Genetic and Evolutionary Computation Conference. 

Munetomo, M. k Goldberg, D. E. (in press). Linkage identification by non-monotonicity detection 
for overlapping functions Evolutionary Computation. 

Osyzcka, A., Tamura, H., k Goldberg, D. E. (in press). A bicriterion approach to constrained opti¬ 
mization problems using genetic algorithms. Evolutionary. Optimization Journal. 

Pelikan, M. k Goldberg, D. E. (in press). Hierarchical problem solving by the Bayesian optimization 
algorithm. GECCO-2000 Proceedings of the Genetic and Evolutionary Computation Conference. 

Pelikan, M., Goldberg, D. E., k Cantii-Paz, E. (in press). Bayesian optimization algorithm, popula¬ 
tion sizing, and time to convergence. GECCO-2000 Proceedings of the Genetic and Evolutionary 
Computation Conference. 

Pelikan, M., Goldberg, D. E., k Cantii-Paz, E. (in press). BOA: The Bayesian Optimization Algo¬ 
rithm. Evolutionary Computation. 

Pelikan, M., Goldberg, D. E., k Lobo, F. (in press). A Survey of optimization by building and using 
probabilistic models. Computational Optimization and Applications. 

Pelikan, M., Goldberg, D. E., k Lobo, F. (in press). A survey of optimization by building and using 
probabilistic models. Proceedings of the 2000 American Controls Conference. 

Pelikan, M. & Lobo, F. (in press). Parameter-less genetic algorithm: A worst-case time and space 
complexity analysis. GECCO-2000 Proceedings of the Genetic and Evolutionary Computation 
Conference. 

Rothlauf, F. k Goldberg, D. E. (in press). Pruefernumbers and genetic algorithms: A lesson how the 
locality of an encoding can harm the performance of GAs. Proceedings of EuroTel 2000. 

Rothlauf, F., Goldberg, D. E., k Heinzl, A. (in press). Bad codings and the utility of well-designed 
genetic algorithms. GECCO-2000 Proceedings of the Genetic and Evolutionary Computation Con¬ 
ference. 

Weile, D. S., Michielssen, E., k Goldberg, D. E. (in press). The compact genetic algorithm: A litmus 
test for genetic algorithm applicability. Proceedings of the IEEE Workshop on Electromagnetic 
Systems. 

5.3 Published 

Cantu-Paz, E, k Goldberg, D. E. (1997). Predicting speedups of ideal bounding cases of parallel 
genetic algorithms. Proceedings of the Seventh International Conference on Genetic Algorithms, 
113-120. 

Cantu-Paz, E, k Goldberg, D. E. (1997). Modeling idealized bounding cases of parallel genetic 
algorithms. Genetic Programming 1997: Proceedings of the Second Annual Conference, 353-361. 
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Cantii-Paz, E. (1998). Designing efficient master-slave parallel genetic algorithms. Genetic Program¬ 
ming 1998: Proceedings of the Third Annual Conference, 455. 

Cantii-Paz, E. (1998). A survey of parallel genetic algorithms. Calculateurs Paralleles, Reseaux et 
Systems Repartis, 10{2), 141-171. 

Cantii-Paz, E. (1998). Using Markov chains to analyze a bounding case of parallel genetic algorithms. 
Genetic Programming 1998: Proceedings of the Third Annual Conference, 456-462. 

Cantii-Paz, E. (1999). Migration policies and takeover times in genetic algorithms. GECCO-99 
Proceedings of the Genetic and Evolutionary Computation Conference, 775. 

Cantii-Paz, E. (1999). Topologies, migration rates, and multi-population parallel genetic algorithms, 
GECCO-99 Proceedings of the Genetic and Evolutionary Computation Conference, 91-98. 

Cantii-Paz, E., & Goldberg, D. E. (1999). On the scalability of parallel genetic algorithms. Evolu¬ 
tionary Computation, 7 (4), 429-449. 

Giguere, P., & Goldberg, D. E. (1998). Population sizing for optimum sampling with genetic algo¬ 
rithms: A case study of the onemax problem. Genetic Programming 1998: Proceedings of the 
Fourth Annual Conference, 496-503. 

Goldberg, D. E. (1999). Prom genetic and evolutionary optimization to the design of conceptual 
machines. Evolutionary Optimization Journal, i(l), 1-12. 

Goldberg, D. E. (1999). A meditation on the application of genetic algorithms. In Y. Rahmat-Samii 
& E. Michielssen, Electromagnetic optimization by genetic algorithms (pp. xxi-xxvi). 

Goldberg, D. E. (1999). The race, the hurdle, and the sweet spot: Lessons from genetic algorithms 
for the automation of innovation and creativity. In P. Bentley, Evolutionary design by computers. 
San Francisco: Morgan Kaufmann (pp. 105-118). 

Goldberg, D. E. (1999). Using time efficiently: Genetic-evolutionary algorithms and the continuation 
problem. GECCO-99 Proceedings of the Genetic and Evolutionary Computation Conference, 212- 
219. 

Goldberg, D. E., & O’Reilly, U. (1998). Where does the good stuff go, and why? How contextual 
semantics influence program structure in simple genetic programming. Proceedings of the First 
European Workshop on Genetic Programming. 

Goldberg, D. E., & Wang, L. (1998). Adaptive niching via coevolutionary sharing. In D. Quagliarella, 
J. Periaux, C. Poloni, and G. Winter (Eds.) Genetic algorithms and evolution strategies in engi¬ 
neering and computer science (pp. 21-38). Chichester: Wiley. 

Goldberg, D. E., & Voessner, S. (1999). Optimizing global-local search hybrids. GECCO-99 Proceed¬ 
ings of the Genetic and Evolutionary Computation Conference, 220-228. 

Harik, G., Cantii-Paz, E., Goldberg, D. E., & Miller, B. L. (1999). The gambler’s ruin problem, 
genetic algorithms, and the sizing of populations. Evolutionary Computation, 7(3), 231-254. 

Harik, G., & Lobo, F. G. (1999). A parameter-less genetic algorithm. GECCO-99 Proceedings of the 
Genetic and Evolutionary Computation Conference, 258-265. 
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6 Interactions and Transitions 

This section lists meeting participation, presentations, and transitions. 

6.1 Meeting participation and presentation, 1998-1999 

During project year 1998-1999 Professor Goldberg gave invited lectures as follows: 

Chevron Lecture Series. A six-hour series of lectures entitled Simplifying complexity: Competent 
genetic algorithms as a model of and mechanism for innovating in complex systems was given in the 
Bay area and distributed through the company on videotape. 

Templeton Seminar. Participated in the Templeton workshop on Accelerating Creativity, and gave 
the lecture “The Design of Innovation: Lessons from Genetic and Evolutionary Computation.” 

IIT Madras. Was invited to India and gave a series of four lectures at IIT Madras with attendance 
from governmental agencies and corporate representatives in addition to campus attendance. 

HollandFest. Was invited to make a presentation at a Festschift honoring John Holland’s 70th 
birthday and gave the presentation “An Integrated, Effective Theory of Building Block Difficulty.” 

GECCO 99. Chaired GECCO 99 and gave two talks, one on “Optimizing global-local search hybrids” 
and another on “Using time efficiently: Genetic-evolutionary algorithms and the continuation problem.” 

IWSCI 99. Shared stage with Lofti Zadeh at International Workshop on Soft Computing in Industry 
in Japan and gave the talk “The Race, the Hurdle, and the Sweet Spot: Genetic Algorithms as a 
Computational Model of Innovation.” 

EP99. Participated in Electronic Prototyping 99 at WPAFB, giving the talk “Competence and Effi¬ 
ciency in Genetic Algorithm Design: What it Means to Real Users.” 

Expert Systems 98. Gave the keynote address at Expert Systems 1998 at Cambridge University 
and gave the talk, “The Race, the Hurdle, and the Sweet Spot: Three Lessons from Genetic Algorithm 
Design for the Understanding of Human Innovation. 

Professor Goldberg’s affiliates and students were active presenters and participants during project 
year 1998-1999 as well: 

1. E. Cantu-Paz presented “Using Markov Chains to analyze a bounding case of parallel genetic 
algorithms” and Designing efficient master-slave parallel genetic algorithms (a poster) at GP 98 
in Madison, Wisconsin. 
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2. Phillipe Giguere presented “Population sizing for optimum sampling with genetic algorithms at 
GP 98, Madison, WI. 

3. Fernando Lobo presented “Compressed introns in a linkage learning genetic algorithm” at GP 
98, Madison, WI. 

4. Ole Mengshoel presented “Deceptive and other functions of unitation as Bayesian networks” at 
GP 98, Madison, WI. 

5. E. Cantu-Paz presented “ Migration policies and takeover times in genetic algorithms” as a poster, 
“Topologies, migration rates, and multi-population parallel genetic algorithms” as a full paper, 
and he also chaired a workshop on Parallel GAs and gave an invited tutorial on parallel GAs at 
GECCO-99 in Orlando, FL. 

6. Fernando Lobo presented “ A parameter-less genetic algorithm” at GECCO-99 as a full paper. 

7. Masaharu Munetomo presented “A genetic algorithm using linkage identification with mono¬ 
tonicity detection” at the 1999 IEEE Conference on Systems, Man, & Cybernetics in Japan, and 
“Identifying linkage groups by nonlinearity/non-monotonicity detection” at GECCO-99. 

8. Martin Pelikan presented “BOA: The Bayesian Optimization Algorithm” at GECCO-99 as a full 
paper. 

6.2 Meeting peirticipation and presentation, 1997—1998 

During the project year 1997-98, Professor Goldberg gave invited seminars at Dortmund University, 
GMD (Bonn), Daimler Benz (Berlin). Technical University of Berlin, Leiden University, Utrecht Uni¬ 
versity, the University of Central England (Birmingham). UCLA (Los Angeles). USC (Los Angeles), 
UCSD (La Jolla, California), Stanford University, University of Paris VI, Dassault Aviation (Paris), 
and . Additionally he gave invited tutorial lectures at Eurogen 98 in Trieste, Italy, Hydroinformatics 
98 in Copenhagen, Denmark, and Genetic Programming 98 in Madison, Wisconsin. 

Professor Goldberg also gave a keynote lecture at Hydroinformatics 98 in Copenhagen and invited 
lectures at Navigating Complexity (Portland, Oregon) and EUROGEN 98 (Trieste, Italy). 

Professor Goldberg’s affiliates and students were active presenters and participants as well: 

1. E. Cantu-Paz presented “Using Markov Chains to analyze a bounding case of parallel genetic 
algorithms” and Designing efficient master-slave parallel genetic algorithms (a poster) at GP 98 
in Madison, Wisconsin. 

2. Phillipe Giguere presented “Population sizing for optimum sampling with genetic algorithms at 
GP 98, Madison, WI. 

3. Fernando Lobo presented “Compressed introns in a linkage learning genetic algorithm” at GP 
98, Madison, WI. 

4. Ole Mengshoel presented “Deceptive and other functions of unitation as Bayesian networks” at 
GP 98, Madison, WI. 
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6.3 Meeting participation and presentation, 1997 

During the first project year, Professor Goldberg gave invited lectures at Osaka Institute of Technology, 
Osaka Kyoiku University, Hitachi Corporation, the Tokyo Institute of Technology, the Japan Society 
of Mechanical Engineers Meeting, the IMA Workshop on Evolutionary Computation (U. Minnesota), 
Los Alamos National Laboratory, the Rowland Institute for Science, and the MIT AI Laboratory. He 
also presented “Modeling idealized bounding cases of parallel genetic algorithms.” at CP 97. 
Professor Goldberg’s affiliates and students were active presenters and participants as well: 

1. E. Cantu-Paz presented “Predicting speedups of ideal bounding cases of parallel genetic algo¬ 
rithms” at ICGA 97 in East Lansing, Michigan. 

2. G. Harik presented “Learning Linkage” at FOG A IV in San Diego and “The gambler’s ruin 
problem, genetic algorithms, and the sizing of populations” at ICEC 97 in Indianapolis. 

3. J. Horn presented “Genetic algorithms (with sharing) in search, optimization, and machine learn¬ 
ing” at FOGA IV in San Diego. 

4. F. Lobo presented “Decision making in a hybrid genetic algorithm” at ICEC 97 in Indianapolis. 

5. Y. Sakamoto presented “Takeover time in noisy environments” at ICGA 97. 

6.4 Transitions 

EvoNews Article. The Pi’s research and views on the field were highlighted in an article in the 
influential newsletter of the European EvoNet, a network of European researchers on genetic algorithms. 

CMAME Journal. The PI edited a special edition of the influential journal Computational Methods 
in Applied Mechanics and Engineering devoted to genetic and evolutionary computation. The issue 
contains 19 articles on theory, implementation, and application with the lion’s share being applications 
to engineering. 

GECCO-99. The PI chaired the 1999 Genetic and Evolutionary Computation Conference held in 
Orlando, Florida from July 13-17, 1999. 619 people attended the conference with 15 workshops, 25 
tutorials, a 1000-page proceedings. Next year’s conference will be held in Las Vegas (www.genetic- 
algorithm.org). 

GA course online. Pi’s GA course (GE 485) is now being offered online via the web this semester. 
8 off-campus and 25 on-campus students are taking the course, and we expect the off-campus atten¬ 
dance to scale up following the pilot semester. Credit can be obtained through UIUC or the National 
Technological University (NTU). 

Application with Caterpillar. Following a short course given to 25 engineers at Caterpillar in Peoria 
by the PI, PI contracted to apply GAs to a cooling system design problem. 

Telecommunications problems. A visitor from German, Franz Rothlauf (University of Bayreuth) 
and the PI worked on solving some telecommunications problems using random keys codings. 

GA-based companies. There are growing numbers of GA-based companies in the US and around 
the world using technology affected by my work. Optimax, a GA-based scheduling software firm has an 
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integrated product for factory scheduling; Optimax recently sold out for close to $60M to 12, another 
scheduling software firm. The Schema Company in Israel has a GA-based software product named 
Orca for scheduling container ships that is used by a substantial fraction of the container fieet, Silicon 
Biology in Minnesota has developed a data-entry software package with GA-based technology at its 
core. Many large and small financial services firms are using GAs to make portfolio and trading 
decisions. 

Increase in on-campus applications. PI participated in a larger number of applications PhD theses 
in applications areas, including traffic control, spacecraft trajectory optimization, electromagetic design, 
environmental plume detection, molecular design to name a few. 

Real biology. Using GA-like techniques, Jay Mittenthal and PI will examine some basic questions in 
evolutionary biology with algorithms that resemble messy GAs. 

Pipe network optimization. A center in England has spawned a company called Optimal Solutions 
that uses structured messy genetic algorithms to optimize water pipelines. Of course, mGAs trace their 
lineage through AFOSR-sponsored work. 

Battlefield courses of action. A student project in PPs GA course in 1997 turned into a major 
focus of Army work in the generation and analysis of battlefield courses of action. I have joined the 
Army Federated Laboratory project centered at Illinois to work on GA improvements of the growing 
Fox system pioneered by Stu Schlabach. 

Working group on genetic and evolutionary computation. The PI has started the Illinois 
Working Group on Genetic and Evolutionary Computation (IWGGEC) at UIUC to try to gain leverage 
on the growing work at Illinois in this field. At present, there are over 26 faculty with some involvement 
in the field. The hope is that intellectual and joint funding initiatives can build ideas and realize synergy 
and economy among Illinois efforts. 

Electromagnetic applications. PI continued to work with E. Michielssen and Dan Weile here at 
the University of Illinois on applying multiobjective GAs to the solution of antenna and absorber 
design. Professor Michielssen is under contract to AFOSR with a project “Electromagnetic Scattering 
from Complex Structures.” Recent studies have concentrated on carrying over the compact GA and 
extended compact GA of Harik to real electromagnetics problems. 

US News Article. The PPs research and views on the field were highlighted in an article on genetic 
and evolutionary computation in US News and World Reports (July 27, 1998). The article highlighted 
the transition that is now taking place from the laboratory to the real world. 

Technology transfer in financial applications. Ole Mengshoel, a project-affiliated UIUC graduate 
student worked as an intern for First Quadrant, a company that manages large portfolios of stocks for 
institutional clients using genetic algorithms. Lab technology was transfered to a working system and 
more such interaction is planned. 

Technology transfer in hydroinformatics. The PPs keynote address at Hydroinformatics 98 led 
to a number of invitations to consult with software firms in hydraulic and hydrologic applications. A 
private short course was held to transfer the latest GA techniques to a company in California. 

Pipe network optimization. A center in England has spawned a company called Optimal Solutions 
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that uses structured messy genetic algorithms to optimize water pipelines. Of course, mGAs trace their 
lineage through AFOSR-sponsored work. 

Machine tool fault detection. Two students in my GA class working with UIUC faculty Dick DeVor 
developed and published a machine-tool fault detection system that uses a GA to determine whether 
parameters from a drill bit are out of range and should be flagged as a fault. This work has led to a 
PhD-level investigation of GA usage in this problem area. 

Self-assembling molecules. A material science student Milan Keser flnished a PhD under the 
direction of noted material scientist Sam Stupp to predict the conformation of certain self-assembling 
molecules. Results axe were excellent and have been published in prominent scientific journals. 

7 New discoveries, inventions, or patent disclosures 

Although much of the work under this project could be classified as new discoveries or inventions, no 
patent applications, filings, or other attempts to register intellectual property were made this past year. 

8 Honors and Awards 

This section lists awards for the PI and his students. 

8.1 PI honors and awards 

Professor Goldberg was on sabbatical for the academic year 1997-98 during which he continued his 
AFOSR work. He was named a Gambrinus Fellow at the University of Dortmund for the fall semester 
1997 and was named a visiting scholar at Stanford Medical Informatics. 

Professor Goldberg received the 1996 ASEE Wickenden Award for best paper in the Journal of 
Engineering Education for his paper entitled “Change in Engineering Education: One Myth, Two 
Scenarios, and Three Foci.” 

Professor Goldberg was named an Associate of the Center for Advanced Study (CAS) at the Univer¬ 
sity of Illinois for the fall semester 1995 and was released of teaching duties for that semester. During 
the CAS semester, he worked on AFOSR projects and wrote a portion of a manuscript entitled The 
Design of Innovating Machines: Lessons from Genetic Algorithms. 

Professor Goldberg was named an NSF Presidential Young Investigator in 1985. 

8.2 Student honors and awards 

Erick Cantu-Paz received a renewal award for his Fulbright Fellowship. 
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